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This conference paper discusses the role ot the 
Planning, Programing, and Budgeting System (FIBS) in planning tor 
educational programs. Proper PPPS analysis includes (1) correct 
problem definition, (1) consideration ct alternative solutions, (3) 
ascertainment cf peripheral effects ot the alternatives, and (4) 
estimations cf the ccst and effectiveness of tne alternatives 
throughout the probable life of the program. A PPPS analyzer should 
have (1) tnorcugh knowledge of the educational system and an 
appreciation for the complex interactions d?ong various parts ot the 
systerr, and (2) a rational, objective, intellectual approach with a 
large measure ct c err men sense. (Author/! LF) 
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ANAL Y SIS OF EDUCATION A L PROGRAMS 

k 

Polly Carpenter 

The Rand Corporation, Santa Monica, California 



In a Planning, Programming, and Budgeting System, the Program Struc- 
ture summarizes the overall objectives of the school system. These ob- 
jectives are extrains tifcufcionsl in character; that is, they express what 
the schools are trying to accomplish for their clients--their students 
anu the community that supports them. (Presumably, if these objectives 
adequately express the needs and desires of the clients, efforts directed 
merely to keeping the system alive can be minimized.) At the same time, 
it must be possible to identify the resources and activities that the 
school system provides with the primary aim of attaining each objective-- 
that is, to group resources and activities "by objective" in the Program 
Structure. Thus, such object rves must'be more highly aggregated t-han 
the usual behavioral objectives , to which it would be burdensome indeed 
to assign resources and activities. In addition, the resulting data 
would be too detailed and too massive to provide the information needed 
for making decisions at the higher administrative levels wnere programs 
and program elements are if concern. (A program or program element is 
a system of resources and processes that produce something of. use outside 

the program. A behavioral objective is a subobjective within a program 
kk 

element . ) 

The process of analysis within a Planning, Programming, and Budget- 
ing System generates information that describes educational programs. 
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Any views expressed in this paper are those of the author. They 
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For a more detailed discussion of these points, see Program 
Budgeting for School District Planning: Co>icepts and Applications , 

S. A . Hc.ggart, et al. , The Rand Corporation, RM-6116-RC, November 1969 
(especially Ch. II) . 
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rhis information serves two major purposes. The most fundamental is to 
describe on-going programs — what they are, what resources they require, 
and what is their effectiveness. The second purpose served is to facil- 
itate rational comparisons of alternative ways to conduct educational 
programs . 

A key aspect of the analysis required to describe on-going programs 
is a description of what the program actually is — what people, facilities, 
equipment, and materials arc really used and how they are used to attain 
the objectives. The process is analogous to determining actual class 
size in a given school by gathering data on the number of students in 
the classes of interest rather than by using some average pupil-teacher 
ratio for the school or the district. Without this kind of detailed, 
knowledgeable analysis, a Program Budget i9 merely a reorganization of 
the usual budgetary figures; such a budget can easily be misleading be- 
cause it appears to present relationships between resource use and effec- 
tiveness that may not exist. 

If the Program Budget realistically describes the resources required 
by existing programs and their effectiveness, it can be invaluable in 
helping administrators and other decisionmakers to decide how to allocate 
resources among programs. This can result in a better alignment with 
the decisionmakers* judgment as to the proper emphasis for the particular 
student population and community. (For example, in poorer neighborhoods, 
reading and vocational education might be more heavily emphasized than 
in neighborhoods that are more well-to-do.) 

In addition, in the process of formulating the descriptions of the 
programs, data will be generated that will suggest how resources may be 
shifted from one program to another. Such shifts will be necessary if 
a fixed overall budgetary level must be maintained and If a desired im- 
provement in the effectiveness of a program demands additional resources. 
In this sense, programs "compete" tor resources. For example, if addi- 
tional emphasis is needed in the reading program, there rhould be data 
that show how much money can be saved by cutting down, say, certain ex- 
tracurricular activities and that estimate what this money can buy in 
terms of increased resources devoted to reading. 

On the other hand, programs tray reinforce one another. The reading 
program may bolster student work in other academic areas, the student 
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health program may contribute to improved student performance in class 
and in sports, and counseling and guidance may improve the students' 
motivation and thereby their performance in all school activities. In 
general, the relationships among program resources are much easier to 
estimate than are relationships among program effectiveness. Experi- 
mental evidence may be needed to substantiate the latter and should be 
obtained wherever possible so that the effect of shifts in the effec- 
tiveness of one program on the effectiveness of others can be considered 
when decisions must be made. 

In addition to providing data and information describing cn-going 
programs, the analytical approach required by a Program Budgeting System 
cm supply the means to compare alternative ways to meet the objectives 
of a single program. This is the major function of the analytical ef- 
fort. In this role, tho analysis can: 

o Help assess the relative worth of several innovative approaches 
to attaining the same educational outcome (such as improvement 
in reading achievement) 

o Determine whether a single program is becoming more or less ef- 
fective or costly as time passes so that steps may be taken to 
improve it, if necessary 

o Help assess the relative worth of the same program for different 
student populations or in different school settings. 

The goal of the analysis is not to provide the planner with the alterna- 
tive that "maximizes" or "minimizes" specific characteristics; the goal 
is to provide information to which the planner can apply his judgment 
in order to choose the alternative that best meets his :«.eods within his 
constraints, such as budget level or community pressures. 

The analysis of the resource requirements, cost, and effectiveness 
of on-going programs is the base upon which the Program Budgeting System 
must be built. Decisions concerning desirable shifts In resources among 
programs will be supported by this analysis but must, ultimately, repre- 
sent the decisionmakers' Subjective value judgments concerning the de- 
sirable emphasis among programs. More rigorous analysis can be applied 
to alternatives for the same program, however, because here the educa- 
tional outcomes are more directly comparable. This latter type of 
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analysis is usually oriented to specific educational problems such as 
improvement in academic achievement in a particular area or dropout 
prevention. 

Two step-; are crucial to good analysis directed toward assisting 
in the choice among alternatives: (1) problem definition and (2) def- 

inition of alternative means for solution. (See the shaded areas of 
Fig. 1.) Obviously, the right solution to the wrong problem is of 
little value. For example, for a long time the prevention of dropout 
from junior high and high schools was seen as a major problem in public 
education, A possible solution to this problem could be to rigorously 
enforce school attendance laws and to turn guidance ana counseling ser- 
vices into antidropout squads. More recently, however, there has been 
a growing awareness that school dropout may be a symptom of a more seri- 
ous problem, a lack of perception of the value of staying in school, 
which may itself stem from a true lack of return to the dropout popula- 
tion for completed schooling. Thus, a mere decrease in dropout may i.ot 
represent a true gain either for the students or the schools. 

The second requirement for good analysis is for the definition of 
alternative means for solution to the problem (Al, A2, etc., on Fig. 1). 
The proposal of a single solution for a problem in education is rarely 
sufficient because' 

o A single solution gives the decisionmaker no feeling for whether 
he could du better or worse in some other way; the analysis takes 
on the aspect of a M sales pitch" 

o A single solution gives the decisionmaker no opportunity to exer- 
cise his judgment as to the relative worth of various aspects of 
the proposal 

o A single solution is less likely to uncover additional features 
that the decisionmaker had not thought of but would find desir- 
able . 

To make this thesis a little less abstract, consider the process of 
shopping for an automobile. If only one car existed within each price 
range, the buyer would hive to be satisfied with the manufacturer’s judg- 
ment as to the relative worth of styling, economical operation, high per- 
formance, safety features, and so on. But the existence of several cars, 
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each of which may fall within what the buyer can afford and each of 
which represents a different balance among characteristics, permits the 
buyer to choose that car which best fits his value structure. In addi- 
tion, the descriptions of the different cars ir^y suggest features (de- 
sirable or undesirable) that the buyer had not thought of before he 
started shopping. 

I submit that "shopping" for educational programs is at least as 
complex as shopping for automobiles. And because of the complexity of 
problems in education, it is naive to try to provide the decisionmaker 
with the solution that maximizes or minimizes some aspect of crucial 
concern. In fact, most currently fashionable formulas for the cost- 
effectiveness of educational program? are quite frankly window-dressing 
and have little relationship to the programs they purport to describe. 

(I even saw one for the cos t-per-pupil to attain a year's growth in 
reading that was projected backwards from 1 year's data to describe the 
past 10 years. Worse, the black students who were having difficulty in 
reading had been in the system for only 2 of those 10 years.) 

Returning to the shaded areas of Fig. 1, note that the analysib of 
alternatives requires descriptions of their resource requirements, their 
effectiveness, and the processes that relate the two aspects. The non- 
quantifiable considerations include not only those aspects of effective- 
ness (such as improved community relations) that are difficult to measure, 
but also the value judgments that apply relative rankings to all aspects 
cf effectiveness, quantifiable and nonquantifiable. 

Now let us leave this rather general discussion to consider an 
example, drawn from an actual situation but considerably reworked for 
the purposes of this paper. The problem was that a large population of 
Mexican- American students at the junior high level were "underachieving" 
in reading and arithmetic. One particular solution was proposed and 
implemented. It was a combination of the alternatives that I shall de- 
scribe and that I shall hypothesize were also implemented, for the sake 
cf Illustration. 

The actual project had three parts: remedial reading and math, 

the study of occupational technology, and the involvement of parents and 
students in special activities. I shall describe each of these briefly. 
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The subject-matter content of the remedial reading and math was no: 
changed from that of the standard 8th grade curriculum. Each was given 
during one of two periods to classes of 15 students each, approximately 
half of the size of a normal class, A diagnostic/prescriptive approach 
was used. Initial diagnosis of reading difficulties was made by means 
of the Durvell reading test, and of math from the profiles of the rtu- 
dents* performance on subtests of the California Achievement Test in 
math . 

Occupational technology was taught through a variety of means. In 
the classroom, gaming and simulation were used with groups of approxi- 
mately 15 students each. This activity was geared to the reading and 
math curricula and took one period every day. The gaming/simulation 
activity, which is how I shall refer to this from now on, was a highly 
structured representation of real-world situations. Students played 
the roles of actual people, such as a park director or a highway engi- 
neer. Each unit was supplemented by a study trip to a facility directly 
related to classroom work. There were about 19 study trips throughout 
the year. Students helped to make the arrangements for the study trips 
by use of a conference phona. 

The third component was the involvement ol parents and students in 
special activities, which I shall refer to as involvement from here on. 
For involvement of the students, there were two study trips of 4 days 
each. For example, one trip was to a beautiful park on the seashore; 
another went to a park in forested mountains. These study trips were 
again very highly structured. They were intended to break down t v * 
stereotyped roles of students and teachers in the classroom and to in- 
volve students in a prolonged and intensive learning experience. It 
was apparent that they accomplished both of these goals. College stu- 
dents were used as team leaders for the learning activities. 

The parents of the students in the program were also involved. Be- 
fore school opened, they were asked to attend a preschool dinner, where 

they were told what the program was to be about and where their consent 

was sought for the students to participate in the program in general 
and in the intensive involvement trips in particular. In addition, the 
parents were invited to all of the study trips that accompanied the 
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gaming/simulation activity, to the intensive involvement trips, and to 

several other dinner meetings throughout the year. At every one of 

\ 

these activities the parents participated along with the students and 
teachers. For example, they played some of the games during the dinner 
meetings, and whereas the students taught their 
the parents took great pride in teaching their 
of the sessions. Finally, the teachers made he 1 
parents during the course of the year to discus 
with their children’s participation in the prog 
each visit had a clear purpose so that the pare 
Figure 2 displays whit the long-term effeq 
be on achievement xn reading if it were contim. 



parents at the beginning, 
children toward the end 
ne visits to all of the 
b some activity connected 
ram. This assured that 
its were at their ease, 
ts of the program might 
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Normal growth, which 



is represented by the dashed line, would be indicated if a student was 
achieving at the 5th-grade level In the 5th grade, at the 6th-grade 



level in the 6th grade, and so forth. An appipximat ion to the rate of 
growth for Mexi can-American students (Oleman^ et al., 1966) is shown 
by the solid line. Because the Coleman report gave reading achievement 
levels only for the 6th, 9th, and 12th grades, the growth rate Vy grade 
can be inferred only very roughly, as indicated. 

The experimental program was intended to raise the growth rate at 
least to normal and. ideally, to provide sufficient initial growth that 
the student could make up for prior years of underachievement. The pro- 
gram succeeded in the 8th grade in raising the students 1 growth ra;c to 
1.3 months per month, as represented by the sharp peak, while In the 9th 
grade the growth rate was 1.1 months j.r month. Students have continued 
to show normal growth, as suggested. 

Another aspect of effectiveness is the effect of changes in one 
achievement measure on other measures not affected by the program di- 
rectly. Logically, one might think that if a student’s performance in 
reading has been improveu, his performance in other subjects would also 
improve, particularly those such as science that require reading skills. 
Therefore, perhaps the program also raised the students 1 achievement in 
science above normal growth, as suggested by the hypothetical upper dash- 
dot line on Fig. 3. But it is also quite possible that the science pro- 
gram suffered by comparison with the experimental activities. that 
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Fig. 2— Long -term growth in reading achievement 
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Fig. 3— Effects of reading achievement gains on 
achievement in science 
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case, the students' performance in science may even have dropped below 
the normally low achievement for this population. Thus, we must measure 
the students' achievement in all areas of interest so that we will know 
what the indirect effects are, if any. 

For the purposes of illvst ration , various combinations of the three 
components I desc r ibed--the remedial reading arid math, the gaming/simula 
tion, and the involvement — were formed, and the cost of each combination 
for various numbers of students was estimated (Carpenter and Haggart, 
September 1969). In this way, we derived what we term "equal-cost al- 
ternatives," that is, alternative programs that cost very nearly the 

same. Each alternative differs from the others in terms of numbers of 

& 

students involved and probably in achievement growth. These differ- 
ences are illustrated on rig. 4. For example, although the gaming/simu- 
lation activity might not induce as much achievement gain as would the 
total program, it could be provided to five tines as many students. It 
might, however, be too close to the dashed line- -indicat ing 1 month per 
month, or normal growth. The dashed line is critical because the State 
of California considers any achievement growth less than this to be un- 
acceptable. This would mean that, although the involvement could be 
given to 90 children for an equal cost, it would not be acceptable if 
the achievement it induced fell below the critical minimum, as suggested 
If the district is not. required to meet a minimum standard in achieve- 
ment gain, it will be possible to trade off achievement gain on the one 
hand and thv number of students reached on the other. This might be an 
important consideration if the schools need visibility. In that case 
the gaming/simulat ion would always be the test choice, because it is the 
least expensive per student. 

The other two alternatives shown may more than meet the minimum re- 
quirement so thac the choice between then would depend on whether one 
felt it more important to provide a higher rate of achievement to f ewer 
student? or a lower rate of achievement to more students. Because th«. 
remedial reading and math program is not particularly innovative, one 



We estimated the relative contributions to achievement growth by 
each of the three component? efter discussing the question with students 
teachers, and program directors. No direct measures are available. 
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Fig. 4— Effectiveness of alternatives 
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might be more interested in the gaming/simulation plus reading and math, 
even though it can be given to only 60 students. 

Now let uj include another measut ‘ oi effectiveness in our analy- 
sis, Two measures of effectiveness — one, growth rate in reading, and 
the other, an index of attitude change for each program- ate plotted 
against number of students on Fig, 5. This index was derived by assum- 
ing that each alternative would induce a change in attitude relative to 
the change induced by the original program. (These estimates were de- 
rived from subjective opinions of people involved with the program.) 

Thus, the index of attitude change for the original program is unity, 
and the other alternatives have indices less than this. As before, 
gaming/simulation alone looks risky because it is not being reinforced 
with backup programs. In addition, it may induce relatively little at- 
titude change; reading and rath look even poorer in this regard, while 
the involvement is too low on achievement gains. Note how the addition 
of the second measure supports the superiority of the combination of 
reading and math with gaming/s imulat ion over reading and math alone. 

Thus, we may want to accept smaller numbers of students and have re- 
inforcing programs, as in the gaming/simulation and reading and math, 
where we buy achievement gain and attitude change for 60 students, or 
we may prefer to buy less of each for 150 students with the gaming/simu- 
lation alone, Which one a decisionmaker chooses will depend upon whether 
he considers gains per student or numbers of students reached more im- 
portant. 

I should like to use the hypothetical example just presented to 
illustrate some of the features of a good analysis within a Program 
Budgeting System. First, although the problem was defined by tne State 
of California to be underachievement in reading and arithmetic, the 



A slight digression at this point will help to explain the ratio- 
nale behind this figure. Analysts have tendency to lump all measur- 
ables in single indices for the sake of simplicity of manipulation and 
presentation. For example, the number of students in each alternative 
program night have been included in the indices. Although this would 
have made for a very pretty picture, it would have been almost impos- 
sible to interpret because too many variables would be combined in a 
single point. As far as possible, it is better to keep measures* that 
are significant in their own right separate. 
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designers of the experimental program recognized that the underlying 
cause might be ignorance in the Mexi can-Ameriean community of the rcle 
that these subjects play in the world of work. Correct problem defini- 
tion was, therefore, attempted and most of the solutions attacked this 
basic difficulty. 

Most of the other aspects of a good analysis, however, can be 
found only in the hypothetical example, not in the real program. 1 do 
not mean to level any special criticism at the program used for analy- 
sis in this paper, however; this situation is almost universal in educa- 
tion at the present time. 

As discussed previously, the consideration of alternative solutions 
is essential to good analysis in education. These alternatives must be 
described with care and as they actually work in practice. Then their 
resource requirements and costs must be generated from the descriptions; 
that is, these data must be built from the bottom up, not from the top 
down. And, finally, some aLtempt should be made to find causal rela- 
tionships between the resources and internal working of the alternatives 
and their effects. 

Because educational activities are complex and are embedded in so- 
cial structures in the school and community, an important aspect of any 
program is its effect on teachers, students, administrators, and others 
involved. In addition, the impacts of the program on other programs in 
the system should be assessed, whether they be resource impacts or 
changes in effectiveness. Thus, a good analysis gives concrete evidence 
that the peripheral effects of the alternatives Were ascertained as well 
as possible. 

The resource requirements and co 3 t of a program over a period of 
several years should oe estimated so that the effects of short-term re- 
quirements fcr special facilities and the like will be seen in perspec- 
tive. The same holds true for considerations of effectiveness, which 
may be only temporary if the Hawthorne effect is responsible. Thus, a 
good analysis estimates the cost and effectiveness of the alternatives 
throughout the probable life of the program. 

Now that I have discussed the characteristics of a good analysis, 
what are the characteristics of a good analyzer-~one who can generate 
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the kinds of information needed? I would list two: 

o A thorough knowledge of the educational system and an apprecia- 
tion for the complex interactions among the various parts of th._ 
system 

o A rational, objective, intellectual approach with a large measure 
of uncommon "common" sense 

The possession of a kit of sophisticated analytical tools if of much less 
importance than these two characteristics In fact, people who know how 
to use sophisticated techniques often tend to apply them whether or not 
they have anything to do with the problem at hand, 
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